a camera identifier operable to identify the cameras having a viewing 
direction within a predetermined angle of the user-selected viewing direction as identified 
cameras; 

a camera characteristic comparer operable to compare at least one camera 
characteristic affecting the image data quality for each identified camera to determine 
differences therebetween; 

a camera selector operable to select one of the identified cameras as a 
selected camera in dependence upon the determined differences; 

an object representation selector operable to select the representation of the 
object generated from the selected camera as a selected representation; and 

a renderer operable to generate image data by rendering an image of the 
three-dimensional computer model in accordance with the user-selected viewing direction, 
in which texture data based on input image data from the selected camera is rendered onto 
the selected representation of the object.-- b _ 

REMARKS 

This application has been reviewed in light of the Office Action dated 
August 29, 2001. Claims 1-116, 118-129, 131-166, 168-179, 181-192, 195-204,207-251 
are presented for examination. Claims 117, 130, 167, 180, 193, 194, 205 and 206 have 
been canceled, without prejudice or disclaimer of the subject matter presented therein. 
Claims 42, 43, 55, 64-66, 100, 101, 131, 132, 164-166, 168, 169, 172, 173, 175-179, 181, 
182, 185-192, 195, 196, 201-204, 207, 208, 210-213, 235, 236, 239 and 244-246 have been 
amended to define more clearly what Applicants regard as their invention. Claims 249- 
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25 1 have been added to provide Applicants with a more complete scope of protection. 
Claims 1, 21-23, 40, 41, 44, 54-56, 63, 64, 76, 84, 85, 99, 102, 116, 118, 129, 133, 149, 
150, 163, 166, 177-179, 187, 188, 191,201-203,210,211,214, 223-226, 232-234 and 
237-251 are in independent form. Favorable reconsideration is requested. 

A Letter Transmitting Corrected Drawings is submitted herewith to make 
corrections to the drawings set out in the form PTO-948 attached to the Office Action. 

Applicants note with appreciation the allowance of Claims 1-54, 56-63, 65- 
116, 118-129, 131-165, 237 and 240-243 and the indication that Claims 16, 167-169, 180, 
196, 208, 216 and 217 would be allowable if rewritten so as not to depend from a rejected 
claim, and with no change in scope. Since the latter claims have been so rewritten, they 
are now believed to be in condition for allowance, have not been so rewritten because, for 
the reasons given below, their base claim is believed to be allowable. 

The Office Action objected to the drawings based on the Draftsperson's 
objection to Figs. 3, 14 and 15, as set forth on form PTO-948. 

First, cancellation of Claims 117, 130, 167, 180, 193, 194, 205 and 206 
renders the rejections of those claims moot. 

Claims 166, 170-173, 175, 177, 179, 183-187, 189-195, 197, 199, 201,203- 
207, 209, 212, 213 and 244-246 were rejected under 35 U.S.C. § 103(a) as being obvious 
from U.S. Patent No. 6,025,847 to Marks in view of U.S. Patent No. 5,182,641 to Diner et 
al and U.S. Patent No. 5,877,779 to Goldberg et al Claims 174, 176, 198 and 200 were 
rejected under 35 U.S.C. § 103(a) as being obvious from Marks in view of Diner et al, 
Goldberg et al, and the Sharp Corporation's Model VC-5W20E Video Cassette Recorder 
("the Sharp VCR"). Claims 178 and 188 were rejected under 35 U.S.C. § 103(a) as 
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obvious from Marks and Diner et al Claims 202, 210 and 21 1 were rejected under 35 
U.S.C. § 103(a) as being obvious from Marks, Diner et al, and U.S. Patent No. 5,587,752 
to Petruchik. Claims 214, 215, 218, 219, 221, 223-227, 230-236 and 247-248 were 
rejected under 35 U.S.C. § 103(a) as being obvious from Marks, Diner et al, Goldberg et 
al and Petruchik, and further in view of applicant's admitted prior art Haralick et al., 
Computer and Robot Vision Volume 2 ("Haralick"), and Wolberg, Digital Image Warping . 
Claims 220 and 222 were rejected under 35 U.S.C. § 103(a) as being obvious from Marks, 
Diner et al, Goldberg et al, Petruchik, Haralick et al, and Wolberg, and further in view 
of the Sharp VCR. 

Claims 178 and 188 were rejected under 35 U.S.C. § 102(b) as being 
anticipated by Diner et al Claims 55 and 239 are rejected under 35 U.S.C. § 102(e) as 
being anticipated by U.S. Patent No. 5,91 1,035 to Tsao. 

Rejection under 35 U.S.C 103(a) 

Claim 191 is directed to a method of processing image data defining a 
plurality of sequences of images, each from a respective camera, of an object moving in a 
scene to produce signals that define a representation of the object in a three-dimensional 
computer model, and to generate image data in accordance with a user-selected viewing 
direction. Claim 191 recites: (a) receiving data to define a user-selected viewing direction, 
(b) calculating the angle between the user-selected viewing direction and the viewing 
direction of each camera, (c) identifying the cameras having a viewing direction within a 
predetermined angle of the user-selected viewing direction, (d) comparing at least one 
camera characteristic affecting image data quality for each identified camera, (e) selecting, 
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based upon the comparison, one of the identified cameras from which to process input 
image data to define a representation of the object in the three-dimensional computer 
model, (f) generating image data by rendering an image of the three-dimensional computer 
model in accordance with the user-selected viewing direction, and (g) input texture image 
data from the selected camera is rendered onto the representation of the object. 

One important feature of processing image data according to Claim 191 is 
that the user can select one of the cameras with superior image data quality from which to 
render an image of a three-dimensional computer model. 

Marks relates to a method of generating a 3D model from an image. The 
source image provided to the system is an image of a static scene showing the static 
physical object to be modeled. The static image is displayed to the user and the user then 
creates a 3D model of the room by creating wireframe primitives that approximate the 
shape, size and location of the objects in the picture (column 3, lines 3-6). Consequently, 
Marks does not teach or suggest the features of Claim 191 in the present application of 
processing image data defining a plurality of sequences of images , each from a respective 
camera, of an object moving in a scene. 

Further, as the Office Action acknowledges in part 10, Marks does not 
disclose generating image data by rendering an image of the three-dimensional computer 
model in accordance with a user-selected viewing direction or at least one camera 
characteristic affecting image data quality. 

Diner et ah relates to a system whereby a plurality of television cameras are 
used to view a robot arm or a remotely operated vehicle controlled by an operator. The 
system provides the user with the ability the select one of the cameras for image display. 
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The Office Action, in part 21, cites Diner et ah as disclosing that the 
representation of an object rendered is determined in dependence upon the user-selected 
viewing direction, the respective viewing directions of cameras, and at least one camera 
characteristic affecting image data quality. To support this argument, the Examiner refers 
to column 1, lines 31-41 of Diner et ah In fact, column 1, lines 31-41 merely discuss the 
potential of multiple camera television systems to provide depth information. 

Goldberg et ah relates to a method for rendering an object or scene from a 
preselected viewpoint onto a display whereby the object is represented by a texture map 
stored in memory of processor-based system, and the viewpoint is represented by 
geometric data stored in memory. 

The Office Action argues, in part 24, that Goldberg et ah discloses "the 
selection of a representation of each object." The Examiner refers to column 1 and column 
2 of Goldberg et ah Applicants maintain that Goldberg et ah is totally silent about there 
being a plurality of representations of each object in a three-dimensional computer model 
and is totally silent about the selection of one of these representations for rendering. 

Applicants submit that nothing in Marks, Diner et ah, and Goldberg et ah, 
would teach or suggest a method, as recited in Claim 191, of: (a) receiving data to define a 
user-selected viewing direction, (b) calculating the angle between the user-selected 
viewing direction and the viewing direction of each camera, (c) identifying the cameras 
having a viewing direction within a predetermined angle of the user-selected viewing 
direction, (d) comparing at least one camera characteristic affecting image data quality for 
each identified camera, (e) selecting, based upon the comparison, one of the identified 
cameras from which to process input image data to define a representation of the object in 
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the three-dimensional computer model, (f) generating image data by rendering an image of 
the three-dimensional computer model in accordance with the user-selected viewing 
direction, and (g) input texture image data from the selected camera is rendered onto the 
representation of the object. 

Accordingly, it is respectfully submitted that Claim 191 is patentable over 
Marks, Diner et al and Goldberg et al, taken individually or in combination. 
Claims 203 and 246 are apparatus claims similar to method Claim 191, and are believed to 
be patentable for at least the same reasons as discussed above in connection with Claim 
191. 

Claims 249, 250 and 251 recite features similar to those discussed above 
with respect to Claim 191 and therefore are also believed to be patentable over Marks, 
Diner et al and Goldberg et al for the reasons discussed above. 

Claim 201 relates to an image processing method in which image data from 
each of a respective sequence of images, each from a different camera, is processed to 
define a representation of at least one object in a three-dimensional computer model. A 
representation of each object is selected for rendering an image according to a user- 
selected viewing direction, the viewing direction of each camera and a plurality of camera 
parameters related to image data quality. Claim 201 recites that the camera parameters are 
tested in a predetermined order, with the selection of a representation for rendering being 
based upon tests to identify a camera parameter that is sufficiently different for the 
cameras. 

Applicants have found no teaching or suggestion in Marks, Diner et al and 
Goldberg et al of an image processing method in which a sequence of images, each from a 
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different camera, is processed to define a representation of at least one object in a three- 
dimensional computer model. A representation of each object is selected for rendering an 
image according to a user-selected viewing direction, the viewing direction of each camera 
and a plurality of camera parameters related to image data quality. Nor does Marks, Diner 
et al and Goldberg et al disclose that the camera parameters are tested in a predetermined 
order, with the selection of a representation for rendering being based upon tests to identify 
a camera parameter that is sufficiently different for the cameras. 

Accordingly, it is respectfully submitted that Claim 201 is patentable over 
Marks, Diner et al and Goldberg et al, taken individually or in combination. 

Petruchik relates to a method for producting a composite photographic 
image from an exposed photographic film and a selected prerecorded image chosen at the 
time of exposure of the photographic image (see column 3, lines 10-13). 

The Office Action, at part 32, argues that Petruchik discloses the feature of 
selecting image data to be used to define an object from a plurality of images of the object 
each recorded by a specific camera. More specifically, Petruchik teaches that a composite 
two-dimensional image is produced from an exposed film frame and a pre-recorded image 
selected bv a user . Applicants believe that Petruchik is totally silent about selecting image 
data from images recorded by a plurality of cameras, to be used to define an object in a 
three-dimensional computer model . 

Accordingly, it is respectfully submitted that Claim 202 is patentable over 
Marks, Diner et al and Petruchik, taken individually or in combination. 

Claims 210 and 21 1 are apparatus claims which relate to similar subject 
matter as that of claims 191, 201, 203 and 246, discussed above. 
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Consequently, for the reasons set out above, we submit that Claims 202, 
210 and 21 1 are allowable over the prior art. 

Claims 214, 215, 218, 219, 221, 223-227, 230-236 and 247-248 were 
rejected under 35 U.S.C. § 103(a) as being obvious from Marks, Diner et al, Goldberg et 
al and Petruchik, and further in view of applicant's admitted prior art Haralick et al., 
Computer and Robot Vision Volume 2 ("Haralick"), and Wolberg, Digital Image Warping . 
Applicants believe that the prior art cited against has been shown in discussion above to 
not address the subject matter. 

The total lack of relevance of Marks , Diner , Goldberg and Petruchik to the 
subject matter of independent claims 214, 223-226, 232-234, 247 and 248 is immediately 
apparent from our discussion of these documents above in relation to claims 191, 201-203, 
210, 21 1 and 246, which concern similar subject matter. 

The Examiner argues in parts 36-39 of the Office Action that the following 
features recited in, for example, independent claim 214 are obvious based on Haralick and 
Wolberg : 

testing whether first and second images of the object displayed from the generated 
image data will be discontinuous by testing whether the image data for the object in 
the second image in the sequence differs by more than a predetermined amount 
from predetermined image data; and 

if the image data for the object in the second image differs by more than a 
predetermined amount, generating modified image data for the object in the second 
image. 
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The Examiner's reasons for arguing that these features are obvious is that 
the applicant referred to Haralick and Wolberg in the specification describing these 
processing steps. 

However, with respect, Haralick is referred to in the specification as a mere 
example of a technique which could be used to determine the similarity of two images. 

Haralick discloses a number of applications for which the disclosed 
techniques of calculating a similarity measure between images may be useful (see, for 
example, Table 16.1 on page 294 of Haralick) . However, Haralick is totally silent about 
using the techniques to test whether first and second images of an object generated by 
rendering texture data onto a three-dimensional computer model will be discontinuous. 

Consequently, the fact that Haralick was cited by the applicant as a mere 
example of a technique which could be used to perform the recited processing operation 
does not in any way mean that Haralick renders the recited features obvious. 

Similarly, the applicant cited Wolberg as a mere example of a document 
which discloses a processing technique which could be used to produce an image which 
will not appear discontinuous to the user. 

Again, Wolberg does not disclose the application of the technique in the 
context recited in claim 214. Indeed, Wolberg discloses in the first paragraph of part 7.5. 1 
on page 222 that the processing technique may be applied to special effects, such as in the 
creation of a sequence of images showing the transformation of a goat to an ostrich to a 
turtle to a tiger to a women. 

Consequently, it is again immediately clear that Wolberg does not disclose 
or make obvious any of the features recited in claim 214. 
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For the reasons set out above, therefore, it is submitted that independent 
Claims 214, 223-226, 232-234, 247 and 248 are allowable over the prior art. 

A review of the other art of record has failed to reveal anything which, in 
Applicants' opinion, would remedy the deficiencies of the art discussed above, as 
references against the independent claims herein. Those claims are therefore believed 
patentable over the art of record. 

The other claims in this application are each dependent from one or another 
of the independent claims discussed above and are therefore believed patentable for the 
same reasons. Since each dependent claim is also deemed to define an additional aspect of 
the invention, however, the individual consideration or reconsideration, as the case may 
be, of the patentability of each on its own merits is respectfully requested. 

This Amendment After Final Rejection is believed clearly to place this 
application in condition for allowance and its entry is therefore believed proper under 37 
C.F.R. § 1.1 16. At the very least, however, it is believed that the formal rejections have 
been overcome, and cancellation of Claims eliminates all issues relating to those claims. 
Accordingly, In any event, however, entry of this Amendment After Final Rejection, as an 
earnest effort to advance prosecution and reduce the number of issues, is respectfully 
requested. Should the Examiner believe that issues remain outstanding, he she is 
respectfully requested to contact Applicants 1 undersigned attorney in an effort to resolve 
such issues and advance the case to issue. 

Applicants respectfully request favorable consideration and early passage to 
issue of the present divisional application. 
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In view of the foregoing amendments and remarks, Applicants respectfully 
request favorable reconsideration and early passage to issue of the present application. 

Applicants 1 undersigned attorney may be reached in our New York office 
by telephone at (212) 218-2100. All correspondence should continue to be directed to our 
below listed address. 



Respectfully submitted, 




FITZPATRICK, CELLA, HARPER & SCINTO 
30 Rockefeller Plaza 
New York, New York 10112-3801 
Facsimile: (212)218-2200 
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Attorney Docket No. 01263.000700. 

VERSION WITH MARKINGS TO SHOW CHANGES MADE TO CLAIMS 

42. (Amended) A storage medium storing instructions for causing a 
programmable processing apparatus to become operable to perform a method according to any of 
claims [1 to 22] L21 and 22 . 

43. (Amended) A signal conveying instructions for causing a programmable 
processing apparatus to become operable to p erform a method according to any of claims [1 to 22] 
1.21 and 22 . 

55. (Amended) A method of generating a model of an object in a three- 
dimensional computer model, in which: 

a transformation is applied to image data from a first camera relating to the object and 
its shadow which maps the image data [for one of the object and its shadow] to a surface to give first 
transformed image data : 

a transformation is applied to image data from a second camera relating to the object 
and its shadow which maps the image data [for one of the object and its shadow] to the surface to 
give second transformed image data : [and] 

the first transformed image data and the second transformed image data are compared 
to distinguish transformed image data relating to the object from transformed image data relating 
to its shadow: and 
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the object is modelled in dependence upon [part of] the transformed image data 
relating to the object . 

64. (Amended) An apparatus for generating a model of an object in a three- 
dimensional computer model, comprising: 

means for applying a transformation to image data from a first camera relating to the 
object and its shadow which maps the image data [for one of the object and its shadow] to a surface 
to give first transformed image data : 

means for applying a transformation to image data from a second camera relating to 
the object and its shadow which maps the image data [for one of the object and its shadow] to the 
surface to give second transformed image data ; [and] 

means for comparing the first transformed image data and the second transformed 
image data to distinguish transformed image data relating to the object from transformed image data 
relating to its shadow: and 

means for modelling the object in dependence upon [part of] the transformed image 
data relating to the object . 



65. (Amended) A storage medium storing instructions for causing a 
programmable processing apparatus to become operable to perform a method according to any of 
claims [44 to 55] 44, 54 and 55 . 
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66. (Amended) A signal conveying instructions for causing a programmable 
processing apparatus to become operable to perform a method according to any of claims [44 to 55] 
44. 54 and 55 . 

100. (Amended) A storage medium storing instructions for causing a 
programmable processing apparatus to become operable to perform a method according to any of 
claims [67 to 84] 67 and 84 . 

101. (Amended) A signal conveying instructions for causing a programmable 
processing apparatus to become operable to perform a method according to any of claims [67 to 84] 
67 and 84 . 

131. (Amended) A storage medium storing instructions for causing a 
programmable processing apparatus to become operable to perform a method according to any of 
claims [102 to 1 17] 102 and 116 . 

132. (Amended) A signal conveying instructions for causing a programmable 
processing apparatus to become operable to perform a method according to any of claims [102 to 
117] 102 and 116 . 
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164. (Amended) A storage medium storing instructions for causing a 
programmable processing apparatus to become operable to perform a method according to any of 
claims [133 to 149] 133 and 149 . 

165. (Amended) A signal conveying instructions for causing a programmable 
processing apparatus to become operable to perform a method according to any of claims [133 to 
149] 133 and 149 . 

1 66. (Amended) A method of processing image data defining a sequence of images 
of at least one object moving in a scene to produce signals defining a representation of each object 
in a three-dimensional computer model, and to generate image data by rendering an image of the 
three-dimensional computer model in accordance with a user-selected viewing direction, the method 
comprising: 

processing the image data to identify image data relating to respective objects in the 

scene; 

defining a representation of each object in the three-dimensional computer model in 
dependence upon the identified image data; 

generating image data by rendering an image of the three-dimensional computer 
model in accordance with a user-selected viewing direction in which texture data based on the 
identified image data is rendered onto the object representations; and 

generating [quality information for the image data indicating a quality of the image 
data determined in dependence upon the user-selected viewing direction] reliability information 
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indicating the reliability of the image data in dependence upon the angle between the user-selected 
viewing direction and the viewing direction from which the input image data was recorded . 

1 68. (Amended) A method according to claim [ 1 67] 166, wherein the information 
indicating the reliability is generated in dependence upon a linear relationship between reliability 
[quality] and the angular difference between the user-selected viewing direction and the viewing 
direction from which the input image data was recorded. 

169. (Amended) A method according to claim [167] 166, further comprising the 
step of generating information indicating how to change the viewing direction to improve the 
generated reliability. 

1 72. (Amended) A method according to claim 1 66, wherein the [quality] reliability 
information is generated as pixel data within the generated image data. 

173. (Amended) A method according to claims 166, further comprising the step 
of generating a signal conveying the image data and the [quality] reliability information. 

175. (Amended) A method according to claim 1 66, further comprising the step of 
displaying an image using the generated image data and displaying the [quality] reliability 
information. 
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1 76. (Amended) A method according to claim 1 66, further comprising the step of 
making a recording of the image data and the [quality] reliability information either directly or 
indirectly. 

177. (Amended) A method of rendering an image in accordance with a user- 
selected viewing direction of a three-dimensional computer model comprising a representation and 
associated texture data for at least one object, the texture data being derived from image data 
recorded by at least one camera, the method comprising: 

generating image data by rendering an image of the three-dimensional computer 
model in accordance with a user-selected viewing direction, in which the texture data is rendered 
onto each representation; and 

[generating quality information for the image data indicating a quality of the image 
data determined in dependence upon the user-selected viewing direction] generating reliability 
information indicating the reliability of the image data in dependence upon the angle between the 
user-selected viewing direction and the viewing direction from which the input image data was 
recorded . 

178. (Amended) An image processing method in which object data defining a 
three-dimensional computer model of at least one object in a scene is rendered in accordance with 
a user-selected viewing direction using image data recorded by a camera having a viewing direction 
to render each object, and an indicator of a quality of the generated image data is produced for 
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output to the user in dependence upon the angle between the user-selected viewing direction and the 
viewing direction of the camera . 

179. (Amended) Apparatus for processing image data defining a sequence of 
images of at least one object moving in a scene to produce signals defining a representation of each 
object in a three-dimensional computer model, and to generate image data by rendering an image 
of the three-dimensional computer model in accordance with a user-selected viewing direction, the 
apparatus comprising: 

means for processing the image data to identify image data relating to respective 
objects in the scene; 

means for defining a representation of each object in the three-dimensional computer 
model in dependence upon the identified image data; 

means for generating image data by rendering an image of the three-dimensional 
computer model in accordance with a user-selected viewing direction in which texture data based 
on the identified image data is rendered onto the object representations; and 

means for generating [quality information for the image data indicating a quality of 
the image data determined in dependence upon the user-selected viewing direction] reliability 
information indicating the reliability of the image data in dependence upon the angle between the 
user-selected viewing direction and the viewing direction from which the input image data was 
recorded. 
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181. (Amended) Apparatus according to claim [180] 179, operable to perform 
processing such that the information indicating the reliability is generated in dependence upon a 
linear relationship between [quality] reliability and the angular difference between the user-selected 
viewing direction and the viewing direction from which the input image data was recorded. 

182. (Amended) Apparatus according to claim [180] 179, further comprising 
means for generating information indicating how to change the viewing direction to improve the 
generated reliability. 

185. (Amended) Apparatus according to claim 1 79, operable to perform processing 
such that the [quality] reliability information is generated as pixel data within the generated image 
data. 

1 86. (Amended) Apparatus according to claim 1 79, further comprising means for 
displaying an image using the generated image data and displaying the [quality] reliability 
information. 

187. (Amended) Apparatus for rendering an image in accordance with a user- 
selected viewing direction of a three-dimensional computer model comprising a representation and 
associated texture data for at least one object, the texture data being derived from image data 
recorded by at least one camera, the apparatus comprising: 
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means for generating image data by rendering an image of the three-dimensional 
computer model in accordance with a user-selected viewing direction, in which the texture data is 
rendered onto each representation; and 

means for generating [quality information for the image data indicating a quality of 
the image data determined in dependence upon the user-selected viewing direction] reliability 
information indicating the reliability of the image data in dependence upon the angle between the 
user-selected viewing direction and the viewing direction from which the input image data was 
recorded. 

188. (Amended) An image processing apparatus operable to render object data 
defining a three-dimensional computer model of at least one object in a scene in accordance with 
a user-selected viewing direction using image data recorded by a camera having a viewing direction 
to render each object, and operable to produce an indicator of a quality of the generated image data 
for output to the user in dependence upon the angle between the user-selected viewing direction and 
the viewing direction of the camera . 

1 89. (Amended) A storage medium storing instructions for causing a 
programmable processing apparatus to become operable to perform a method according to any of 
claims [166 to 178] 166. 177 and 178 . 
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190. (Amended) A signal conveying instructions for causing a programmable 
processing apparatus to become operable to perform a method according to any of claims [166 to 
178] 166. 177 and 178 . 

191. (Amended) A method of processing image data defining a plurality of 
sequences of images, each from a respective camera, of [at least one] an object moving in a 
scene to produce signals defining a representation of [each] the object in a three-dimensional 
computer model, and to generate image data by rendering an image of the three-dimensional 
computer model in accordance with a user-selected viewing direction, the method comprising: 

receiving data defining a user-selected viewing direction: 
calculating the respective angle between the user-selected viewing direction and the respective 
viewing direction of each camera: 

identifying the cameras having a viewing direction within a predetermined angle 
of the user-selected viewing direction as identified cameras: 

comparing at least one camera characteristic affecting image data quality for each 
identified camera to determine differences therebetween: 

selecting one of the identified cameras as a selected camera in dependence upon 
the determined differences: 

processing input image data from [at least one] the selected camera to define [at 
least one] a representation of [each] the object in the three-dimensional computer model; and 

generating image data by rendering an image of the three-dimensional computer 
model in accordance with the user-selected viewing direction, in which texture data based on 
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input image data from the selected camera is rendered onto [a] the representation of [each] the 
objectf;]. 

[wherein: the representation of each object rendered is determined in dependence 
upon the user-selected viewing direction, the respective viewing directions of cameras, and at 
least one camera characteristic affecting image data quality.] 



192. (Amended) A method according to claim 191 or claim 249 , wherein , when 
comparing at least one camera characteristic affecting image data quality, at least one of the 
following are compared: [the representation of each object rendered is determined in dependence 
upon the user-selected viewing direction, the viewing direction of respective cameras, and at 
least one of:] the methods of transferring the image data from respective cameras; the resolution 
of respective cameras; the shutter speed of respective cameras; the stability of the image data 
from respective cameras; and whether the image data from respective cameras is colour or black 
and white. 



195. (Amended) A method according to claim 191 or claim 249 , wherein a 
plurality of camera characteristics affecting image quality are compared [considered to determine 
the representation of each object for rendering]. 

196. (Amended) A method according to claim 195, wherein the camera 
characteristics affecting quality are considered in a predetermined order and values for each 
respective camera characteristic are compared, with the [determination] selection of [the 
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representations to be rendered] a camera being made once the tests identify a characteristic 
which differs by more than a predetermined amount for given cameras. 

201 . (Amended) An image processing method in which image data from each 
of a respective sequence of images, each from a different camera, is processed to define a 
representation of at least one object in a three-dimensional computer model, and wherein a 
representation of each object is selected for rendering in dependence upon a user-selected 
viewing direction, the viewing direction of each camera and [at least one] a plurality of camera 
parameters related to image data quality , wherein the plurality of camera parameters are tested in 
a predetermined order, with the selection of a representation for rendering being made once the 
tests identify a camera parameter which is sufficiently different for the cameras . 

202. (Amended) An image processing method in which a user-selected viewing 
direction in accordance with which an image of at least one object in a three-dimensional 
computer model is to be rendered is used to select, from among image data defining a plurality 
of images of the object each recorded by a respective camera, image data to be used to define the 
object in the three-dimensional computer model, the selection being carried out in dependence 
upon the user-selected viewing direction, together with the viewing direction of each camera and 
[at least one] a plurality of camera parameters related to image data quality , wherein the 
plurality of camera parameters are tested in a predetermined order, with the selection of image 
data being made once the tests identify a camera parameter which is sufficiently different for the 
cameras. 
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203. (Amended) An image processing apparatus for processing image data 
defining a plurality of sequences of images, each from a respective camera, of [at least one] an 
object moving in a scene to produce signals defining a representation of [each] the object in a 
three-dimensional computer model, and to generate image data by rendering an image of the 
three-dimensional computer model in accordance with a user-selected viewing direction, the 
apparatus comprising: 

means for receiving data defining a user-selected viewing direction; 

means for calculating the respective angle between the user-selected viewing 
direction and the respective viewing direction of each camera: 

means for identifying the cameras having a viewing direction within a 
predetermined angle of the user-selected viewing direction as identified cameras: 

means for comparing at least one camera characteristic affecting image data 
quality for each identified camera to determine differences therebetween; 

means for selecting one of the identified cameras as a selected camera in 
dependence upon the determined differences; 

means for processing input image data from [at least one] camera to define a [at 
least one] the selected representation of each] the object in the three-dimensional computer 
model; and 

means for generating image data by rendering an image of the three-dimensional 
computer model in accordance with the user-selected viewing direction, in which texture data 
based on input image data from the selected camera is rendered onto [a] the representation of 
[each] the object.[;] 
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[the apparatus being operable to perform processing such that: 

the representation of each object rendered is determined in dependence upon the 

user-selected viewing direction, the respective viewing directions of cameras, and at least one 

camera characteristic affecting image data quality]. 

204. (Amended) Apparatus according to claim 203 or claim 250 . wherein the 
means for comparing at least one camera characteristic affecting image data quality is operable 
to compare [perform processing such that the representation of each object rendered is 
determined in dependence upon the user-selected viewing direction, the viewing direction of 
respective cameras, and] at least one of: the methods of transferring the image data from 
respective cameras; the resolution of respective cameras; the shutter speed of respective cameras; 
the stability of the image data from respective cameras; and whether the image data from 
respective cameras is colour or black and white. 

207. (Amended) Apparatus according to claim 203 or claim 250 , operable to 
perform processing [such that] to compare a plurality of respective camera characteristics 
affecting image quality [are considered to determine the representation of each object for 
rendering], 

208. (Amended) Apparatus according to claim 207, operable to perform 
processing such that the camera characteristics affecting quality are [considered] compared in a 
predetermined order [and values for each respective camera characteristic are compared], with 
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the [determination] selection of [the representations to be rendered] a camera being made once 
the tests identify a characteristic which differs by more than a predetermined amount for given 
cameras. 

210. (Amended) An image processing apparatus operable to process image data 
from each of a respective sequence of images, each from a different camera, to define a 
representation of at least one object in a three-dimensional computer model, and to select a 
representation of each object for rendering in dependence upon a user-selected viewing 
direction, the viewing direction of each camera and [at least one] a plurality of camera 
parameters related to image data quality , the apparatus being operable to test the plurality of 
camera parameters in a predetermined order and to select a representation for rendering when the 
tests identify a camera parameter which is sufficiently different for the cameras . 

211. (Amended) An image processing apparatus operable to use a user-selected 
viewing direction in accordance with which an image of at least one object in a three- 
dimensional computer model is to be rendered to select, from among image data defining a 
plurality of images of the object each recorded by a respective camera, image data to be used to 
define the object in the three-dimensional computer model, the selection being carried out in 
dependence upon the user-selected viewing direction, together with the viewing direction of each 
camera and [at least one] a plurality of camera parameters related to image data quality , wherein 
the apparatus is operable to test the plurality of camera parameters in a predetermined order and 
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to select image data when the tests identify a camera parameter which is sufficiently different for 
the cameras . 

212. (Amended) A storage medium storing instructions for causing a 
programmable processing apparatus to become operable to perform a method according to any of 
claims [191 to 202] 191.20L202 and 249 . 

213. (Amended) A signal conveying instructions for causing a programmable 
processing apparatus become operable to perform a method according to any of claims [191 to 
202] 191.201.202 and 249 . 

235. (Amended) A storage medium storing instructions for causing a 
programmable processing apparatus to become operable to perform a method according to any of 
claims [214 to 225] 214. 223.224 and 225 . 

236. (Amended) A signal conveying instructions for causing a programmable 
processing apparatus to become operable to perform a method according to any of claims [214 to 
225] 214. 223. 224 and 225 . 

239. (Amended) Apparatus for generating a model of an object in a three- 
dimensional computer model, comprising: 
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an image data transformer for applying a transformation to image data from a first 
camera relating to the object and its shadow which maps the image data [for one of the object 
and its shadow] to a surface to give first transformed image data , and for applying a 
transformation to image data from a second camera relating to the object and its shadow which 
maps the image data [for one of the object and its shadow] to the surface to give second 
transformed image data : [and] 

a transformed image data comparer for comparing the first transformed image 
data and the second transformed image data to distinguish transformed image data relating to the 
object from transformed image data relating to its shadow: and 

an object modeller for modelling the object in dependence upon [part of] the 
transformed image data relating to the object . 

244. (Amended) Apparatus for processing image data defining a sequence of 
images of at least one object moving in a scene to produce signals defining a representation of 
each object in a three-dimensional computer model, and to generate image data by rendering an 
image of the three-dimensional computer model in accordance with a user-selected viewing 
direction, the apparatus comprising: 

an image data identifier for processing the image data to identify image data 
relating to respective objects in the scene; 

an object modeller for defining a representation of each object in the three- 
dimensional computer model in dependence upon the identified image data; 
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a Tenderer for generating image data by rendering an image of the three- 
dimensional computer model in accordance with a user-selected viewing direction in which 
texture data based on the identified image data is rendered onto the object representations; and 

a [quality] reliability information generator for generating [quality] reliability 
information [for the image data indicating a quality of the image data determined in dependence 
upon the user-selected viewing direction] indicating the reliability of the image data in 
dependence upon the angle between the user-selected viewing direction and the viewing 
direction from which the input image data was recorded . 

245. (Amended) Apparatus for rendering an image in accordance with a user- 
selected viewing direction of a three-dimensional computer model comprising a representation 
and associated texture data for at least one object, the texture data being derived from image data 
recorded by at least one camera, the apparatus comprising: 

a renderer for generating image data by rendering an image of the three- 
dimensional computer model in accordance with a user-selected viewing direction, in which the 
texture data is rendered onto each representation; and 

a [quality data] reliability information generator for generating [quality] reliability 
information [for the image data indicating a quality of the image data determined in dependence 
upon the user-selected viewing direction] indicating the reliability of the image data in 
dependence upon the angle between the user-selected viewing direction and the viewing 
direction from which the input image data was recorded . 
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246. (Amended) An image processing apparatus for processing image data 
defining a plurality of sequences of images, each from a respective camera, of [at least one] an 
object moving in a scene to produce signals defining a representation of [each] the object in a 
three-dimensional computer model, and to generate image data by rendering an image of the 
three-dimensional computer model in accordance with a user-selected viewing direction, the 
apparatus comprising: 

a data receiver for receiving data defining a user-selected viewing direction; 

an angle calculator operable to calculate the respective angle between the user- 
selected viewing direction and the respective viewing direction of each camera: 

a camera identifier operable to identify the cameras having a viewing direction 
with a predetermined angle of the user-selected viewing direction as identified cameras: 

a camera characteristic comparer operable to compare at least one camera 
characteristic affecting image data quality for each identified camera to determine differences 
therebetween: 

a camera selector operable to select one of the identified cameras as a selected 
camera in dependence upon the determined differences; 

an object representation generator for processing input image data from [at least 
one] camera to define a [at least one] the selected representation of [each] the object in the three- 
dimensional computer model; and 

a renderer for generating image data by rendering an image of the three- 
dimensional computer model in accordance with the user-selected viewing direction, in which 



-xix- 



Application No. 09/229,898 
Attorney Docket No. 01263.000700. 

texture data based on input image data from the selected camera is rendered onto [a] the 

representation of each object[;]. 

[the apparatus being operable to perform processing such that: 

the representation of each object rendered is determined in dependence upon the 

user-selected viewing direction, the respective viewing directions of cameras, and at least one 

camera characteristic affecting image data quality.] 
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